feat: implement adaptive frequency detection and auto-parameter adjustment by Erikpostt · Pull Request #285 · biomarkersParkinson/paradigma

Erikpostt · 2026-04-16T09:34:49Z

Description

This PR implements the foundation for adaptive frequency detection and automatic parameter adjustment across ParaDigMa pipelines. This is based on making ParaDigMa work for lower sampling frequencies, where we noticed that upsampling is inferior to keeping the same frequency but adjusting frequency-based features accordingly.

The changes should not break / change much.

Changes

Frequency Detection: Added detect_sampling_frequency() utility with median-based detection
Config System: Implemented sampling_frequency property setter with auto-update of frequency-dependent parameters
Orchestrator Integration: Added automatic frequency detection after data preparation
Preprocessing: Made resampling_frequency optional (maintains current frequency by default)
Documentation: Updated guides explaining frequency flexibility

Key Features

✅ Backward compatible with existing code

- Add sampling_frequency property with setter to IMUConfig - Implement _update_frequency_dependent_params() virtual method - Override in GaitConfig: auto-clamp spectrum_high_frequency and mfcc_high_frequency to Nyquist - Override in TremorConfig: auto-clamp fmax_peak_search and fmax_mfcc to Nyquist - Add warnings when frequency bounds are clamped for safety - Fixes issue where 50Hz/64Hz data caused 'out of bounds' errors in feature extraction This allows users to set sampling_frequency at any point and freq-dependent params adapt automatically, preventing Nyquist violations without manual config management.

…ss initialization This fixes the initialization order bug where the update method was being called in IMUConfig.__init__() before subclass attributes like mfcc_high_frequency were set. Solution: defer calls until the end of GaitConfig and TremorConfig __init__. The property setter still works for post-initialization frequency changes, and the virtual method pattern correctly updates all bounds when sampling_frequency changes at any point in the config lifecycle. Verified with test_adaptive_frequencies_demo.py - all 3 tests pass.

- Remove test_adaptive_frequencies_demo.py (development validation only) - Remove implementation notes from IMUConfig (internal architectural detail) - Keep core adaptive frequency bounds implementation clean for production Commit validates that all 3 frequency test cases (50/64/100 Hz) pass successfully.

…band

…fix-adaptive-frequency-bounds

Erikpostt · 2026-04-17T04:57:09Z

Couple of notes:

The gait output is intended to change: if you look at the duration_s of 18 seconds, it should belong in 0_20 instead of 20_inf, so this fixed an earlier error.
Gait pipeline adds an additional 1 / fs to end_s, because time represents the start of a window, so end_s represented the start time of the final window, which should be the end time of the final window.
Be critical about how we pose that our pipeline works >= 50 Hz, I'm not sure whether I phrased it honestly and covered the necessary details right.
There were some auto-generated files created by the agent that should be removed.

nienketimmermans

If I understand it correctly, the resampling frequency is set to 100 Hz if it is not provided, but this will result in upsampling of the data if the sampling frequency is below 100 Hz right? Shouldn't it be set to the auto-detected sampling frequency to ensure uniform sampling at that frequency?

…Hz when resampling

Erikpostt · 2026-04-17T09:22:14Z

If I understand it correctly, the resampling frequency is set to 100 Hz if it is not provided, but this will result in upsampling of the data if the sampling frequency is below 100 Hz right? Shouldn't it be set to the auto-detected sampling frequency to ensure uniform sampling at that frequency?

Yes, good catch! I think I've changed it correctly now. I've additionally removed resampling from data preparation (in run_paradigma with skip_preparation=False), which became redundant.

Note that I've now added segmentation to data preparation as this functionality was called inside the resample_data function, which was now removed.

- Add import for numpy and segmenting module functions - Implement auto-segmentation in prepare_raw_data() using create_segments() and discard_segments() - This restores the ability to split non-contiguous data into segments during data preparation - Auto-segmentation is independent of resampling and operates on time gaps - Update docstring to clarify auto-segmentation functionality - All tests now pass

- duration_for_categorization_s: unfiltered parent segment duration - duration_s: actual segment data duration (filtered or unfiltered) Filtered arm swing of 5s in 25s unfiltered segment now correctly: - Categorizes into 20+s (based on parent unfiltered duration) - Sums only 5s to category total (actual filtered data)

… branch Changed duration_for_categorization_s back to duration_unfiltered_segment_s for consistency with release branch naming conventions. This maintains clear structure where both filtered and unfiltered aggregations use the same field for categorization (unfiltered parent segment duration = 25s for 0-20s/20+ determination), while duration_s varies per dataset type (5s for filtered, 25s for unfiltered).

Keep only segment_category for aggregation logic, aligned with category-only approach. The unfiltered duration is no longer stored since categorization is pre-computed and stored in segment_category field. This field will be reintroduced if needed during backward compatibility fallback implementation.

KarsVeldkamp · 2026-04-20T06:56:13Z

 - **Gyroscope**: `deg/s` (degrees per second)

-> ParaDigMa has functionalities for converting [accelerometer](https://biomarkersparkinson.github.io/paradigma/autoapi/paradigma/util/index.html#paradigma.util.convert_units_accelerometer) and [gyroscope](https://biomarkersparkinson.github.io/paradigma/autoapi/paradigma/util/index.html#paradigma.util.convert_units_gyroscope) in other units (e.g., `m/s^2`, `rad/s`) to these standard units. This can also be setup when using `run_paradigma`.
+> ParaDigMa has functionalities for converting [accelerometer](https://biomarkersparkinson.github.io/paradigma/autoapi/paradigma/util/index.html#paradigma.util.convert_units_accelerometer) and [gyroscope](https://biomarkersparkinson.github.io/paradigma/autoapi/paradigma/util/index.html#paradigma.util.convert_units_gyroscope) in other units (e.g., `m/s^2`, `rad/s`) to these standard units. This can also be setup when using run_paradigma`.


Is the ` missing? Or should you also remove the latter one?

KarsVeldkamp · 2026-04-20T06:57:32Z


 > [!NOTE]
-> The specifications above represent **minimally validated requirements**. For example, while ParaDigMa works with accelerometer and gyroscope data sampled at 50 Hz, the effect on processing accuracy has not been empirically validated.
+> The specifications above represent **validated requirements**. ParaDigMa has been tested and verified to work correctly with sampling frequencies as low as 50 Hz, with comparable accuracy to 100 Hz baseline.


If you state that it has been tested and verified to work correctly then the part after the comma becomes redundant right?

KarsVeldkamp · 2026-04-20T08:21:14Z

+        """
+        Update frequency-dependent parameters when sampling_frequency changes.
+
+        Ensures that spectral bounds stay within Nyquist limit (fs/2).


Maybe a minor semantic thing but technically it is called Nyquist frequency right?

KarsVeldkamp

In general this is looking good, but I have a couple of questions before ready to merge

KarsVeldkamp · 2026-04-20T08:24:40Z

+            df=df,
+            segment_nr_colname="data_segment_nr",
+            min_segment_length_s=(
+                min_segment_length_s if min_segment_length_s is not None else 1.5


KarsVeldkamp · 2026-04-20T08:26:41Z

+        logger.info(f"Created {n_segments} segments")
+
+    # Deprecation notice for resampling_frequency parameter
+    if resampling_frequency != 100.0:


What if it is set at 100? Maybe not the most logical thing to do but then the user still expect to already have resampled data right? Or am I not understanding this correctly?

This ensures that the tremor pipeline works well for odd sampling frequencies, since rounding the number of overlapping samples resulted in the creation of too many subwindows

nienketimmermans · 2026-04-23T12:49:36Z

@Erikpostt I have made adjustments to the tremor pipeline so that odd sampling frequencies work well too.

nienketimmermans · 2026-04-23T12:58:11Z

Should we change this to sampling_frequency = int(1/current_dt)? The tremor pipeline currently relies on the sampling frequency being an integer, but not sure where to adjust this

Erikpostt · 2026-04-29T09:22:26Z

I've made some additional changes after sitting down with Kars. In summary:

Documentation updates

Removed manual sampling_frequency setting from examples
Updated to reflect auto-detected sampling frequency during preprocessing
Introduced resampling_frequency as an optional parameter for upsampling/downsampling

Config

Implemented frequency-dependent parameter adaptation
sampling_frequency now auto-detected instead of manually set
Added resampling_frequency parameter for optional resampling (not recommended)

Pipelines updated accordingly

Preprocessing

Moved frequency detection into preprocessing phase (previously in both data preparation and preprocessing)
Cleaner separation of concerns

@nienketimmermans am I correct that segment_length_n should also be an integer? I've changed this to be sure. Also, I'm not sure whether the round_detected_frequency is what we wanted to do for sampling frequencies; I believe you informed me that uneven sampling frequencies should be fine too.

EDIT: this needs a bit more work and testing. Let's discuss this later.

… issues

Erikpostt · 2026-05-08T10:19:02Z

@nienketimmermans if we do automatic resampling, the test data is actually resampled to 101 Hz (instead of 100 Hz) causing quite some changes. Could you check the markdown files and inspect these changes? Perhaps we should be more lenient on rounding (e.g., use only even sampling rates or something similar). What do you think? We can discuss on Monday if needed.

…ng a resmapling_frequency is not recommended

nienketimmermans

Ready to merge!

Erikpostt · 2026-05-12T07:08:25Z

Thank you @nienketimmermans; only awaiting @KarsVeldkamp for approval (upon his return from leave).

KarsVeldkamp · 2026-05-18T07:18:37Z

@Erikpostt The preprocess_ppg_data() function currently modifies the ppg_config object (and similar for imu_config in my own and your pipelines) by calling config._set_sampling_frequency_detected() internally. While this works, for me it makes the intent unclear to callers: it is not obvious that the config object will be mutated. I am not sure what if this is an issue at all but maybe we can discuss this further. This is my only issue with the current adjustments, so apart from this we are ready to merge

…mpling_frequency; Clarified updating of config parameters after resampling;

Erikpostt · 2026-05-18T09:10:15Z

@KarsVeldkamp I've made some changes based on our discussion. Can you verify these changes and perhaps provide some additional suggestions if needed?

KarsVeldkamp

This looks good!

Erikpostt added 8 commits March 20, 2026 17:36

fix: Only clamp MFCC bounds when they exceed Nyquist, preserve 25 Hz …

2cae6b7

…band

fix: Clamp MFCC bounds to exactly Nyquist, not 95% of it

f6c1700

Enforce resampling below 50 Hz

82be156

Merge branch 'main' of github.com:biomarkersParkinson/paradigma into …

45535fe

…fix-adaptive-frequency-bounds

Merge main into branch

4b0ab86

Erikpostt mentioned this pull request Apr 16, 2026

feat: add multi-segment processing support to all pipelines #286

Open

Erikpostt added 4 commits April 16, 2026 16:44

Rebuild docs

52d6eb8

Ensure automatic resampling always runs

cb70384

Ensure automatic resampling always runs

28999c4

Ensure automatic resampling always runs

626b8a5

Erikpostt requested review from KarsVeldkamp and nienketimmermans April 17, 2026 04:54

Erikpostt assigned Erikpostt, nienketimmermans and KarsVeldkamp Apr 17, 2026

nienketimmermans reviewed Apr 17, 2026

View reviewed changes

Removing resampling from data preparation; remove default set to 100 …

1f0b1be

…Hz when resampling

Erikpostt added 4 commits April 17, 2026 11:24

KarsVeldkamp reviewed Apr 20, 2026

View reviewed changes

KarsVeldkamp requested changes Apr 20, 2026

View reviewed changes

Erikpostt and others added 2 commits April 20, 2026 11:29

Ensure sampling frequency is integer with correct rounding

8cb585c

remove rounding for spectrogram computation

236667f

This ensures that the tremor pipeline works well for odd sampling frequencies, since rounding the number of overlapping samples resulted in the creation of too many subwindows

nienketimmermans reviewed Apr 23, 2026

View reviewed changes

Erikpostt added 2 commits April 29, 2026 11:15

Merge changes from tremor pipeline

a43f0a2

Merge changes from tremor pipeline

12ceb0a

Erikpostt added 4 commits April 29, 2026 12:45

Add automatic resampling to tutorials

e6319f3

Resolve merge conflicts

d27bd5c

Removed configuration of imu object in gait pipeline; resolved pytest…

7a456f0

… issues

Resolve imu_config object initialization in gait pipeline

0224fa9

Updated documentation to specify that sampling is automatic and setti…

d972b07

…ng a resmapling_frequency is not recommended

nienketimmermans approved these changes May 12, 2026

View reviewed changes

Fixed incorrect updating of parameters after explicitly defining resa…

cb59eb9

…mpling_frequency; Clarified updating of config parameters after resampling;

Erikpostt requested a review from KarsVeldkamp May 18, 2026 09:10

KarsVeldkamp approved these changes May 18, 2026

View reviewed changes

Erikpostt merged commit 13aafd7 into main May 18, 2026
1 check passed

Erikpostt deleted the feature/adaptive-frequency-foundation branch May 18, 2026 09:23

Conversation

Erikpostt commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changes

Key Features

Uh oh!

Erikpostt commented Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nienketimmermans left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Erikpostt commented Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KarsVeldkamp Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

KarsVeldkamp Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

KarsVeldkamp Apr 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KarsVeldkamp left a comment

Choose a reason for hiding this comment

Uh oh!

KarsVeldkamp Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

KarsVeldkamp Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

nienketimmermans commented Apr 23, 2026

Uh oh!

nienketimmermans Apr 23, 2026

Choose a reason for hiding this comment

Uh oh!

Erikpostt commented Apr 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Erikpostt commented May 8, 2026

Uh oh!

nienketimmermans left a comment

Choose a reason for hiding this comment

Uh oh!

Erikpostt commented May 12, 2026

Uh oh!

KarsVeldkamp commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Erikpostt commented May 18, 2026

Uh oh!

KarsVeldkamp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Erikpostt commented Apr 16, 2026 •

edited

Loading

Erikpostt commented Apr 17, 2026 •

edited

Loading

nienketimmermans left a comment •

edited

Loading

Erikpostt commented Apr 17, 2026 •

edited

Loading

KarsVeldkamp Apr 20, 2026 •

edited

Loading

Erikpostt commented Apr 29, 2026 •

edited

Loading

KarsVeldkamp commented May 18, 2026 •

edited

Loading